Digital video coding involves various processes such as, but not limited to, transform, quantization, motion estimation, in-loop deblocking filtering and entropy coding on the residue of the motion compensated or the intra interpolated block. These processes are implemented with respective devices. To achieve better coding efficiency, the block-based DPCM (Differential Pulse Code Modulation) technique is applied to these processes in video standards such as H.264/AVC.
Image quality in digital video is measured by subjective quality and objective quality. Subjective quality is measured by how a viewer perceives the image and the viewer can tell which video has a better image quality by comparing the artifacts that the viewer finds in the video. Using the block-based DPCM technique, blocking artifacts are introduced into a video image and need to be alleviated. The process of alleviating blocking artifacts is generally known to be deblocking. In a block-based technique, video pixels are encoded block-by-block through video compression. These blocks are inverse transformed and decoded to reconstruct the image.
Objective quality is measured by parameters such as Power Signal to Noise Ratio (PSNR). The more noise exists in a video image, the lower the PSNR and less desirable it will be. There are various types of noise such as Gaussian noise, quantization noise and it is desirable to suppress the noise.
To improve the subjective quality, an in-loop deblocking filter is designed to alleviate blocking artifacts in the video coding standard H.264/AVC. One way to design such an in-loop deblocking filter is to use a bank of predefined low-pass filters. The low-pass filters are described in List, P.; Joch, A.; Lainema, J.; Bjontegaard, G.; Karczewicz, M., Adaptive Deblocking Filter, IEEE Trans. CSVT, Vol. 13, No. 7, 2003. The low-pass filters assume the smooth image model and are capable of denoising blocking artifacts. But, a smooth image model is not always applicable. A video image may contain many singularities, such as edges, textures, etc., and the low-pass filters are incapable of handling these singularities properly as the low-pass filters smooth both blocking artifacts and these singularities.
Another way to design such an in-loop deblocking filter is to use a nonlinear bilateral filter. A nonlinear bilateral filter is described in C. Tomasi. R. Manduchi, Bilateral filtering for gray and color images, Proceedings of IEEE Int. Conf. Computer Vision, 1998 and is designed to address the limitation of low-pass filters in handling singularities because the nonlinear bilateral filter is effective in denoising both blocking and ringing artifacts while retaining the sharpness of singularities such as edges. However, both low-pass filters and nonlinear bilateral filters are not frequency-selective and are incapable of optimally suppressing Gaussian noise.
To improve objective quality, a Wiener filter is used to suppress Gaussian noise, blurring and distortion in a video. A Wiener filter also provides further applications such as improving the coding efficiency of the video coding standard H.264/AVC by determining interpolation filter coefficients at the half-pixel level or quarter-pixel level in motion estimation or motion compensation modules to achieve better image prediction. However, a Weiner filter is incapable of alleviating the blocking artifacts and fails to improve the subjective image quality.
There is a need in the art for an in-loop filter which optimizes both subjective quality and objective quality.